Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add ratelimiting #345

Merged
merged 19 commits into from Jul 31, 2019
Merged

Add ratelimiting #345

merged 19 commits into from Jul 31, 2019

Conversation

paramsingh
Copy link
Collaborator

@paramsingh paramsingh commented Jun 3, 2019

Deployment steps

@paramsingh paramsingh marked this pull request as ready for review June 3, 2019 17:27
@paramsingh paramsingh force-pushed the add-ratelimiting branch 2 times, most recently from fa00500 to bd24f62 Compare June 3, 2019 17:46
Copy link
Collaborator

@alastair alastair left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

some small improvements to documentation and the limit setter script

manage.py Outdated
@click.argument('per_ip')
@click.argument('per_token')
@click.argument('window_size')
def set_rate_limits(per_ip, per_token, window_size):
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we don't have token auth in AB, so how about we remove this parameter (and optionally set it to the same as per_ip?)
Please add documentation and types to the arguments.
It'd make sense to also print what the parameters are set to when you run this function.
Do we want any sanity checking to make sure that these settings aren't set to anything really strict? e.g. a warning if the effective limit goes lower than 1 per second


The AcousticBrainz API is rate limited via the use of rate limiting headers that
are sent as part of the HTTP response headers. Each call will include the
following headers:
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What's the default rate limit if we don't make any changes? It'd be nice to say this here. "We typically set the limit to 10 queries every 10 seconds, but these values may change. Make sure you check the response headers if you want to know the specific values"

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looking at the BU code it seems like the default is 30 queries in 10 seconds. Is this actually useful for us - should we check the mean number of queries per IP address for AB to ensure that this actually results in a reduction of queries, or see if we need to reduce the default.
Our custom defaults should be in the config file instead of hard-coded.

webserver/testing.py Show resolved Hide resolved

bp_core = Blueprint('api_v1_core', __name__)


@bp_core.route("/<uuid:mbid>/count", methods=["GET"])
@crossdomain()
@ratelimit()
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd like to see at least one test to see that we're returning ratelimiting headers on one of these methods

manage.py Outdated Show resolved Hide resolved
manage.py Outdated Show resolved Hide resolved
webserver/views/api/legacy.py Outdated Show resolved Hide resolved
webserver/views/api/v1/test/test_core.py Show resolved Hide resolved
@alastair alastair merged commit 4b89e6e into master Jul 31, 2019
@paramsingh paramsingh deleted the add-ratelimiting branch July 31, 2019 18:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
3 participants